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Introduction 


What Are PIRLS and ePIRLS? 


The Progress in International Reading Literacy Study (PIRLS) is an international assessment of 
student performance in reading literacy at the fourth grade. PIRLS measures students in the fourth 
year of formal schooling 1 because this is typically when students' learning transitions from a focus on 
learning to read to a focus on reading to learn. PIRLS is a collaborative effort between participating 
countries and the International Association for the Evaluation of Educational Achievement (IEA). 2 
Administered every 5 years since 2001, PIRLS 2016 marks the survey’s fourth administration. The 
current administration also provides the first results for ePIRLS: a computer-based extension of PIRLS 
designed to assess students’ comprehension of online information. In 2016, some 58 education 
systems 3 participated in the PIRLS assessment at the fourth year of formal schooling and 16 of these 
systems also participated in ePIRLS. 

What PIRLS and ePIRLS Measure 

PIRLS provides a comprehensive picture of students’ reading literacy achievement. Reading literacy is 
defined as follows: 

Reading literacy is the ability to understand and use those uuritten language forms required 
by society and/or valued by the individual. Readers can construct meaning from texts in a 
variety of forms. They read to learn, to participate in communities of readers in school and 
everyday life, and for enjoyment (Mu 11 is and Martin 2015, p. 12). 

The PIRLS framework focuses on two key content domains: purposes for reading and processes of 
comprehension. There are two key purposes for reading: reading for literary experience and reading 
to acquire and use information. Reading for literary experience involves reading for interest or 
pleasure using texts that emphasize characters, plot events, settings, and themes. Reading to acquire 
and use information includes reading to learn, typically using informational texts that tell students 
about themselves and the world around them. The assessment also integrates four comprehension 


1 For ease of presentation, throughout this report, student participants are referred to as fourth-grade students. 

2 PIRLS 2016, and its partner assessment ePIRLS, were sponsored by the IEA and implemented by the TIMSS & PIRLS 
International Study Center at Boston College. 

3 The term “education system” refers to IEA member countries and benchmarking participants. IEA member “countries” 
may be complete, independent political entities or nonnational entities that represent a portion of a country (e.g., England, 
Hong Kong, the Flemish community of Belgium). Nonnational entities are indicated by italics in the tables and figures, with 
the three-letter international abbreviation for their country following their name. Nonnational entities that are represented 
by their larger country in the main results (e.g., Abu Dhabi in the United Arab Emirates, Ontario in Canada), or whose 
countries are not IEA members (Buenos Aires) are designated as “benchmarking participants.” Benchmarking participants 
are included in a separate section in the tables and figures. For convenience, this report uses the generic term “education 
systems” when summarizing results. PIRLS was administered in 61 education systems; this report does not include the 
results of three education systems that did not administer PIRLS at the target grade. 
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processes within both of the purposes for reading. The four processes of comprehension consist 
of the ability to (1) focus on, and retrieve, explicitly stated information, (2) make straightforward 
inferences, (3) interpret and integrate ideas and information, and (4) evaluate, and critique, content 
and textual elements. These four comprehension processes are presented through two combined 
measures: retrieving and straightforward inferencing and interpreting, integrating, and evaluating. 


The new ePIRLS is an innovative, computer-based assessment of online reading. It is designed to 
measure students’ approaches to informational reading in an online environment. As webpages 
become a more common source for acquiring information, ePIRLS provides measures of students’ 
online reading skills and competencies. Example tasks include identifying a specific webpage, 
filtering content on the webpage for the most relevant information, relating information across 
websites, and judging the credibility of information on the website. Exhibit 1 illustrates the framework 
for reading literacy used for PIRLS and ePIRLS. 


Exhibit 1. Percentages of the PIRLS and ePIRLS assessment items devoted to each reading purpose 
and comprehension process in the PIRLS framework: 2016 


Purposes for Reading PIRLS ePIRLS 


Percentages devoted to 
each domain 


Domain skills 


Literary experience 

£3 Literary passages with questions addressing theme, plot, characters, and 
setting 

Acquire and use information 

B3 Informative or instructional passages with questions addressing definitions 
or facts 


Processes of comprehension 


Retrieving and straightforward inferencing 


ca 

Focus on and retrieve explicitly stated information: Locate and understand 
text related to a question posed 

20 

20 

m 

Make straightforward inferences: Move beyond stated material and connect 
pieces of information 

30 

30 


Interpreting, integrating, and evaluating 


ca 

Interpret and integrate ideas and information: Integrate personal knowledge 
with explicit contextual information 

30 

30 

ca 

Evaluate and critique content and textual elements: Critically assess the 
text from a personal or objective view 

20 

20 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy 
Study (PIRLS), 2016. 
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The PIRLS assessment provides students with short reading passages and accompanying items to 
assess understanding. These items are standardized measures that use multiple-choice and open- 
ended formats. The ePIRLS assessment provides students with passages or a series of web pages 
on a desktop or laptop computer and requires students to use a mouse to navigate the assessment. 
The ePIRLS simulated web pages consisted of two tasks. In the tasks, a teacher avatar guides 
students through the ePIRLS assignments, prompting the students with questions about the online 
information. For both PIRLS and ePIRLS 2016, multiple-choice items were worth one point, whereas 
the value of open-ended items (in which students wrote or typed responses) depended on the depth 
of understanding required: short-answer items were worth 1 or 2 points, while extended response 
items were worth up to 3 points. 

Reporting PIRLS and ePIRLS 2016 Results 

This report summarizes performance on PIRLS and ePIRLS 2016 from a U.S. perspective. PIRLS 
results are based on nationally representative samples of fourth-graders. The international data 
reported for PIRLS 2016 in this report cover 58 countries or other education systems, including the 
United States. Education systems that did not administer PIRLS at the target grade are not included 
in this report; see the international report for their results (Mullis et al. 2017a, 2017b). 

Scale scores and percentiles 

Average scores on PIRLS and ePIRLS are reported on an overall reading scale ranging from Oto 
1,000, with a fixed scale centerpoint of 500 and a standard deviation of 100. The PIRLS reading 
achievement scale was established in PIRLS 2001 based on the achievement distribution across 
all countries or education systems that participated in 2001. The results of each successive 
administration of PIRLS have been placed on the same scale so that scores are comparable 
across years. The ePIRLS online informational reading scale summarizes fourth-grade students’ 
performance in a simulated online environment. The ePIRLS results are reported on the same scale 
as PIRLS. However, ePIRLS assesses how well students comprehend material and navigate content in 
an online environment, using content tabs, links, and icons. The PIRLS scale provides a more general 
measure of reading comprehension and purposes for reading. 

For PIRLS, average subscale scores are also reported for the individual domain skills in the two 
content domains: purposes for reading and processes of comprehension (exhibit 1). However, direct 
comparisons should not be made between subscale scores because the scaling of PIRLS data is 
conducted separately for the combined overall reading scale and each of the subscales. 

The distribution of student achievement is reported using percentiles. Percentile scores identify the 
threshold (or cut) scores along the scale that correspond to the lowest 10 percent, lowest quarter, 
highest quarter, and top 10 percent of students (i.e., the 10th, 25th, 75th, and 90th percentiles). 
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International benchmarks 

PIRLS international benchmarks provide a way to interpret the scale scores and to understand how 
student proficiency varies at different points on the scale. Each successive point, or benchmark, 
is associated with the knowledge and skills that students successfully demonstrate at each level. 
PIRLS describes four levels: Advanced, High, Intermediate, and Low. Exhibit 2 describes the skills 
and score cut-point for each PIRLS benchmark. 


Exhibit 2. Description of PIRLS international reading benchmarks: 2016 


Benchmark j Description of benchmark skills 'i 

(score cut-point) 1 

Advanced 

(625) 

CQ Students can interpret story events and character actions to describe reasons, motivations, 
feelings, and character development with full text-based support in relatively complex literary 
texts. They can begin to evaluate the effect on the reader of the author’s language and style 
choices. 

CQ Students can distinguish and interpret complex information from different parts of text and 
provide full text-based support when reading relatively complex informational texts. They 
can integrate information across a complex informational text and explain relationships and 
sequence activities. They can begin to evaluate visual and textual elements to consider the 
author’s point of view. 

High 

(550) 

CQ Students can locate and distinguish significant actions and details embedded across rela¬ 
tively complex literary texts. They can make inferences to explain relationships between 
intentions, actions, events, and feelings, and give text-based support. They can interpret and 
integrate story events and character actions, traits, and feelings as they develop across the 
text. They can recognize the use of some language features (e.g., metaphor, tone, imagery). 

CQ Students can locate and distinguish relevant information within relatively complex tables or 
informational texts. They can make inferences about logical connections to provide explana¬ 
tions and reasons. They can integrate textual and visual information to interpret the rela¬ 
tionship between ideas and evaluate and make generalizations about content and textual 
elements. 

Intermediate 

(475) 

CQ Students can independently locate, recognize, and reproduce explicitly stated actions, 

events, and feelings when reading a mix of simpler and relatively complex literary texts. They 
can make straightforward inferences about the attitudes, feelings, and motivations of main 
characters. They can begin to recognize language choices and interpret obvious reasons and 
causes, recognize evidence, and give examples. 

CQ Students can locate and reproduce two or three pieces of information from a mix of simpler 
and relatively complex informational texts. They can make straightforward inferences to pro¬ 
vide factual explanations and begin to interpret and integrate information to order events. 


Low CQ Students can locate and retrieve explicitly stated information, actions, or ideas when reading 

(400) relatively simple literary texts. They can make straightforward inferences about events, and 

begin to interpret story events and central ideas. 


CQ Students can locate and reproduce explicitly stated information from relatively simple 
informational texts and other formats and begin to make straightforward inferences about 
explanations, actions, and descriptions. 

NOTE: Score cut-points for the international benchmarks are determined through scale anchoring. Scale anchoring involves select¬ 
ing benchmarks (scale points) on the achievement scales in terms of student performance and then identifying items that stu¬ 
dents scoring at the anchor points answer correctly. The score cut-points are set at equal intervals along the achievement scales. 
The score cut-points were selected to be as close as possible to the standard percentile cut-points (i.e., 90th, 75th, 50th, and 25th 
percentiles). More information on how the score cut-points were set can be found in the PIRLS technical documentation at https:// 
timssandpirls.bc.edu/publications/pirls/2016-methods.html . 


SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy 
Study (PIRLS), 2016. 
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Exhibit 3 describes the skills and score cut-point for each ePIRLS benchmark. 


Exhibit 3. Description of ePIRLS international reading benchmarks: 2016 


^ Benchmark T" 
(score cut-point) 


Description of ePIRLS benchmark skills | 

Advanced 

(625) 

B3 

Students can make inferences from relatively complex online informational texts to support 
an explanation. They can interpret and integrate information within and across web pages 
with interactive features to explain relationships and show thorough understanding. Students 
can evaluate the effects of textual, visual, and interactive elements and begin to consider the 
writer’s point of view. 

High 

(550) 

£3 

Students can make inferences to distinguish relevant information and provide comparisons 
when reading and viewing relatively complex online informational texts. They can interpret 
and integrate information within and across web pages with interactive features to provide 
examples and make contrasts. They can evaluate how graphic elements and language choices 
support content. 

Intermediate 

(475) 

ca 

Students can locate and reproduce information presented in various forms, including indepen¬ 
dent use of navigation features, when reading and viewing relatively complex online informa¬ 
tional texts. They can make straightforward inferences to recognize reasons and actions. They 
can interpret and integrate information across a web page to recognize causes, comparisons, 
and explanations. Students can begin to evaluate the use of interactive features to convey 
information. 

Low 

(400) 

ca 

Students can locate and reproduce explicitly stated information from web pages in relatively 
complex online informational texts that contain text and a variety of dynamic, navigable fea¬ 
tures (e.g., timelines, pop-up boxes). They can begin to make straightforward inferences about 
descriptions. 


NOTE: Score cut-points for the international benchmarks are determined through scale anchoring. Scale anchoring involves 
selecting benchmarks (scale points) on the achievement scales in terms of student performance and then identifying items that 
students scoring at the anchor points answer correctly. The score cut-points are set at equal intervals along the achievement 
scales. The score cut-points were selected to be as close as possible to the standard percentile cut-points (i.e., 90th, 75th, 50th, 
and 25th percentiles). More information on how the score cut-points were set can be found in the PIRLS and ePIRLS technical 
documentation at https://timssandpirls.bc.edu/publications/pirls/2016-methods.html . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy 
Study (PIRLS), 2016. 

For more PIRLS 2016 results, visit the NCES PIRLS website (https://nces.ed.gov/surveys/pirls) , 
the PIRLS website (https://www.pirls.org) , or the TIMSS & PIRLS International Study Center website 
( https://timssandpirls.bc.edu/pirls2016/ ). 


5 











Selected Findings 


U.S. PIRLS Performance 

■ The U.S. overall average reading score was 549 (table 1). This score was higher than the 
PIRLS scale centerpoint, which is set at 500 points. The U.S. overall average reading score 
was lower than the averages for 12 education systems, higher than the averages for 30 
education systems, and not significantly different from the averages for 15 education 
systems. 

■ In 2016, some 16 percent of U.S. fourth-graders performed at or above the Advanced 
(625) benchmark, and 53 percent of fourth-graders performed at or above the High (550) 
benchmark (figure 1). The percentages of U.S. fourth-graders performing at or above 

the Advanced and High benchmarks were higher than the international median. Seven 
education systems (Singapore, the Russian Federation, Northern Ireland-GBR, Ireland, 
Poland, England-GBR, and Moscow City-RUS) had a higher percentage of fourth-graders 
performing at or above the Advanced benchmark than the United States. For the Low (400) 
benchmark, 96 percent of U.S. fourth-graders performed at or above the benchmark, and 
24 education systems had a smaller percentage of students performing at or above the 
benchmark. 

■ U.S. fourth-graders scored higher, on average, than the PIRLS scale centerpoint across all 
four reading subscales in 2016 (table 2). The U.S. average score for each subscale ranged 
from 543 to 557. Seven education systems scored higher on average than the United States 
on every subscale: the Russian Federation, Singapore, Ireland, Northern Ireland-GBR, 
Poland, Finland and Moscow City-RUS. 

■ In 2016, among U.S. fourth-graders, females scored higher on average on the overall 
reading scale than males (553 vs. 545) (figure 4). Compared to the U.S. overall average 
reading score, White and Asian fourth-graders scored higher on average, while Black and 
Hispanic fourth-graders scored lower on average. 

■ U.S. fourth-graders in public schools with less than 50 percent of students eligible for free or 
reduced-price lunch scored higher on average than the U.S. overall average reading score, 
while fourth-graders in public schools with more than 75 percent of students eligible for free 
or reduced-price lunch scored lower on average than the U.S. overall average reading score 
(figure 4). 
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Selected Findings 


Performance over time 

■ The overall average reading scores increased in 11 education systems between the first 
administration of PIRLS (in 2001) and 2016 (figure 2). Since the last administration of 
PIRLS (in 2011) and 2016, overall average reading scores increased in 10 education 
systems. Overall average reading scale scores increased during both of these intervals for 
six education systems: the Russian Federation, Hungary, Italy, Slovenia, Norway, and Quebec- 
CAN. In five education systems, the overall average reading score decreased since the last 
administration. 

■ There was no measurable change in the U.S. overall average reading scale score between 
2001 (542) and 2016 (549) (figures 2 and 3). Between 2011 and 2016, the overall average 
reading score for U.S. fourth-graders declined from 556 to 549. However, the U.S. overall 
average reading score in 2016 remained higher than the U.S. overall average reading score 
in 2006 (540) (figure 3). 

■ At the 25th percentile, U.S. fourth-graders scored lower in 2016 (501) than in 2011 (510) 
(figure 3). The average scores for U.S. fourth-graders at the other percentiles were not 
measurably different between 2011 and 2016. 

U.S. ePIRLS Performance 

■ In the new 2016 ePIRLS online assessment, the United States scored above the ePIRLS 
scale centerpoint, set at 500 (table 3). U.S. fourth-graders’ ePIRLS online informational 
reading average score was 557, which was lower than the averages for 3 education systems, 
higher than the averages for 10 education systems, and not measurably different from the 
averages for 2 education systems. 

■ In 2016, some 18 percent of U.S. fourth-graders scored at or above the Advanced 
benchmark in online informational reading, placing the United States above the international 
median, along with five other education systems (figure 5). A larger percentage of fourth- 
graders in one country, Singapore (34 percent), met or exceeded th e Advanced benchmark 
compared to the United States (18 percent). Additionally, 56 percent of U.S. fourth-graders 
scored at or above the High benchmark in online informational reading. 

■ In 2016, among U.S. fourth-graders, females scored higher on average on the online 
informational reading scale than males (560 vs. 554) (figure 6). Compared to the U.S. 
average online informational reading scale score (557), White and Asian fourth-graders 
scored higher on average, while Black and Hispanic fourth graders scored lower on average. 

■ U.S. fourth-graders in public schools with less than 50 percent of students eligible for free 
or reduced-price lunch scored higher on average than the U.S. online informational reading 
average scale score, while fourth-graders in public schools with more than 75 percent of 
students eligible for free or reduced-price lunch scored lower on average than the U.S. online 
informational reading average scale score (figure 6). 
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PIRLS Tables and Figures 


Table 1. PIRLS overall reading average scale scores of fourth-grade students, by education system: 2016 


Education system 

Overall average 
scale score 

Education system 

Overall average 
scale score 

PIRLS scale centerpoint 1 

500 


PIRLS scale centerpoint 1 

500 

Russian Federation 

581 

o 

Spain 

528 ▼ 

Singapore 2 

576 

o 

Belgium (Flemish)-BEL 

525 ▼ 

Hong Kong-CHN 2 

569 

o 

New Zealand 

523 ▼ 

Ireland 

567 

o 

France 

511 ▼ 

Finland 

566 

o 

Belgium (French)-BEL 2 

497 ▼ 

Poland 

565 

o 

Chile 

494 ▼ 

Northern Ireland-GBR 

565 

o 

Georgia 2 

488 ▼ 

Norway 

559 

o 

Trinidad and Tobago 

479 ▼ 

Chinese Taipei-CHN 

559 

o 

Azerbaijan 

472 ▼ 

England-GBR 

559 

o 

Malta 2 

452 ▼ 

Latvia 2 

558 

o 

United Arab Emirates 

450 ▼ 

Sweden 

555 


Bahrain 

446 ▼ 

Hungary 

554 


Qatar 

442 ▼ 

Bulgaria 

552 


Saudi Arabia 

430 ▼ 

United States 2 

549 


Iran, Islamic Republic of 

428 ▼ 

Lithuania 

548 


Oman 

418 ▼ 

Italy 

548 


Kuwait 

393 ▼ 

Denmark 2 

547 


Morocco 

358 ▼ 

Macao-CHN 

546 


Egypt 

330 ▼ 

Netherlands 2 

545 


South Africa 

320 ▼ 

Australia 

544 




Czech Republic 

543 


Benchmarking education systems 


Canada 2 

543 


Moscow City-RUS 

612 O 

Slovenia 

542 


Madrid-ESP 2 

549 

Austria 2 

541 

▼ 

Quebec-CAN 2 

547 

Germany 

537 

V 

Ontario-CAN 

544 

Kazakhstan 

536 

V 

Andalusia-ESP 

525 ▼ 

Slovak Republic 

535 

▼ 

Dubai-UAE 

515 ▼ 

Israel 2 

530 

V 

Buenos Aires-ARG 

480 ▼ 

Portugal 2 

528 

V 

Abu Dhabi-UAE 

414 V 


© Score is higher than U.S. average score. 

▼ Score is lower than U.S. average score. 

1 The PIRLS scale centerpoint is set at 500 points and represents the mean of the overall achievement distribution in 2001. The PIRLS scale is the 
same in each administration; thus a value of 500 in 2016 equals 500 in 2001. 

2 See appendix table A-l for details on sampling issues in these education systems. 

NOTE: Education systems are ordered by overall average scale score. Italics indicate participants identified and counted in this report as a nonnational 
entity that represents a portion of a country. All average scores reported as higher or lower than the U.S. average score are different at the .05 level 
of statistical significance. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between 
the United States and one education system may be significant, while a large apparent difference between the United States and another education 
system may not be significant. The standard errors of the estimates are shown in table 1 available at https://nces.ed.gov/pirls/pirls2016/pirls2016 
tableOl.asp . Education systems that did not administer PIRLS at the target grade are not shown; see the international report for their results. 

Five education systems participated in PIRLS Literacy (Egypt, Iran, Kuwait, Morocco, and South Africa); two of these education systems completed 
both PIRLS and PIRLS Literacy (Iran and Morocco). More detail on PIRLS Literacy is available in the technical notes at https://nces.ed.gov/pirls/ 
pirls!6technotes.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 


8 













PIRLS Tables and Figures 


Figure 1. Percentage of fourth-grade students reaching the PIRLS international benchmarks in reading, by education system: 
2016 


Education system 




Percent of students reaching each international benchmark 

Advanced 

(625) 

High 

(550) 

Intermediate 

(475) 

Low 

(400) 






29 O 

66 O 

89 O 

97 






26 O 

70 © 

94 O 

99 O 






22 © 

61 O 

87 O 

97 






21 O 

62 © 

89 O 

98 O 






20 O 

61 O 

89 O 

98 O 






20 O 

57 O 

86 

97 






19 

55 

83 

95 






18 

65 © 

93 O 

99 O 






18 

62 © 

91 O 

98 O 






17 

56 

85 

97 






16 

53 

83 

96 






16 

51 

81 

94 ▼ 






15 

58 © 

90 O 

99 © 






14 

59 © 

90 O 

98 O 






14 

57 

88 O 

98 O 






14 

57 

90 O 

99 O 






13 ▼ 

46 ▼ 

75 ▼ 

91 ▼ 






13 ▼ 

50 

83 

96 






12 ▼ 

52 

86 

97 






11 ▼ 

49 

83 

96 






11 ▼ 

52 

86 

97 






11 ▼ 

41 ▼ 

73 ▼ 

90 ▼ 






11 ▼ 

47 ▼ 

81 

95 






11 ▼ 

52 

87 O 

98 © 






10 ▼ 

47 ▼ 

81 

93 ▼ 






10 ▼ 

49 

85 

97 






10 ▼ 

50 

86 O 

98 O 






8 ▼ 

47 ▼ 

84 

98 O 






8 ▼ 

48 ▼ 

88 O 

99 © 






7 ▼ 

42 ▼ 

84 

98 O 






7 ▼ 

38 ▼ 

79 ▼ 

97 






6 ▼ 

39 ▼ 

80 ▼ 

97 






5 ▼ 

20 ▼ 

43 ▼ 

68 ▼ 






4 ▼ 

24 ▼ 

55 ▼ 

80 ▼ 






4 ▼ 

35 ▼ 

80 

97 O 






4 ▼ 

30 ▼ 

72 ▼ 

94 ▼ 






3 ▼ 

25 ▼ 

61 ▼ 

87 ▼ 

Qatar 





3 ▼ 

3 ▼ 

17 ▼ 

22 ▼ 

42 ▼ 

65 ▼ 

66 ▼ 

92 ▼ 






10 ▼ 

47 ▼ 

82 

96 
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20 

40 

60 

80 100 







Percent 







See notes at the end of the table. 
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PIRLS Tables and Figures 


Figure 1. Percentage of fourth-grade students reaching the PIRLS international benchmarks in reading, by education system: 
2016—Continued 


Education system 





Percent of students reaching each international benchmark 

Advanced 

(625) 

High 

(550) 

Intermediate 

(475) 

Low 

(400) 

Georgia 1 






22 ▼ 

60 ▼ 

86 ▼ 

Bahrain 






14 ▼ 

41 ▼ 

69 ▼ 

Oman 






10 ▼ 

32 ▼ 

59 ▼ 

Azerbaijan 






18 ▼ 

54 ▼ 

81 ▼ 

Iran, Islamic Republic of 






11 ▼ 

37 ▼ 

65 ▼ 

Saudi Arabia 






11 ▼ 

35 ▼ 

63 ▼ 

Malta 1 






13 ▼ 

45 ▼ 

73 ▼ 

Kuwait 






6 ▼ 

22 ▼ 

51 ▼ 

Egypt 






3 ▼ 

11 ▼ 

31 ▼ 

Morocco 






3 ▼ 

14 ▼ 

36 ▼ 

South Africa 






2 ▼ 

8 ▼ 

22 ▼ 

International median 






47 ▼ 

82 

96 

Benchmarking education 









systems 









Moscow City-RUS 






84 O 

98 © 

ioo O 

Ontario-CAN 1 






50 

82 

96 

Quebec-CAN 






50 

87 

98 O 

Dubai-UAE 1 






40 ▼ 

69 ▼ 

87 ▼ 

Madrid-ESP 






51 

89 O 

99 O 

Andalusia-ESP 






37 ▼ 

78 ▼ 

97 

Buenos Aires-ARG 






20 ▼ 

55 ▼ 

83 ▼ 

Abu Dhabi-UAE 





-1-1 2 T 

11 ▼ 

31 ▼ 

55 ▼ 


1 

0 

20 

40 

60 

80 100 







Percent 






| Advanced | High ■ Intermediate | Low ■ Below Low 
© Percentage is higher than the U.S. percentage at the same benchmark. 

▼ Percentage is lower than the U.S. percentage at the same benchmark. 

# Rounds to zero. 

1 See appendix table A-l for details on sampling issues in these education systems. 

NOTE: Education systems are ordered by the percentage of students reaching the Advanced international benchmark. Italics indicate participants 
identified as a nonnational entity that represents a portion of a country. All percentages reported as higher or lower than the U.S. percentage are 
different at the .05 level of statistical significance. The tests for significance take into account the standard error for the reported difference. Thus, 
a small difference between the United States and one education system may be significant, while a large apparent difference between the United 
States and another education system may not be significant. The international median represents all participating PIRLS education systems, includ¬ 
ing the United States. The international median represents the percentage at which half of the education systems have that percentage of students 
at or above the median and half have that percentage of students below the median; benchmarking participants are not included in the internation¬ 
al median. The standard errors of the estimates are shown in figure 1 available at https://nces.ed.gov/pirls/pirls2016/pirls2016 figureOl.asp . Edu¬ 
cation systems that did not administer PIRLS at the target grade are not shown; see the international report for their results. Five education systems 
participated in PIRLS Literacy (Egypt, Iran, Kuwait, Morocco, and South Africa); two of these education systems completed both PIRLS and PIRLS 
Literacy (Iran and Morocco). More detail on PIRLS Literacy is available in the technical notes at https://nces.ed.gov/pirls/pirlsl6technotes.asp . 
SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 
2016. 
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PIRLS Tables and Figures 


Table 2. PIRLS purposes for reading and processes of comprehension average subscale scores of fourth-grade students, by 
education system: 2016 


Purposes for reading_ _ Processes of comprehension 


Education system 

Literary experience 

Acquire and use 
information 

Retrieving and 
straightforward 
inferencing 

Interpreting, 
integrating, and 
evaluating 

PIRLS scale centerpoint 1 

500 

500 

500 

500 

Russian Federation 

579 O 

584 O 

581 O 

582 O 

Singapore 2 

575 O 

579 O 

573 O 

579 O 

Ireland 

571 © 

565 © 

566 O 

569 O 

Northern Ireland-GBR 

570 0 

561 © 

562 © 

567 O 

Poland 

567 O 

564 O 

560 O 

570 O 

Finland 

565 O 

569 O 

572 O 

562 O 

England-GBR 

563 

556 © 

556 O 

561 

Hong Kong-CHN 2 

562 

576 © 

568 O 

568 O 

Norway 

560 

559 © 

561 O 

558 

Hungary 

558 

551 

552 

557 

United States 2 

557 

543 

543 

555 

Sweden 

556 

555 © 

560 O 

553 

Latvia 2 

555 

561 O 

554 © 

562 

Bulgaria 

551 

554 O 

550 

552 

Denmark 2 

551 

543 

550 

546 ▼ 

Italy 

549 ▼ 

549 

547 

550 

Chinese Taipei-CHN 

548 ▼ 

569 O 

560 © 

558 

Lithuania 

547 ▼ 

551 

549 

548 

Canada 2 

547 ▼ 

540 

541 

545 ▼ 

Australia 

547 ▼ 

543 

541 

549 

Netherlands 2 

546 ▼ 

545 

546 

544 ▼ 

Czech Republic 

545 ▼ 

541 

551 O 

538 ▼ 

Austria 2 

544 ▼ 

539 

550 

534 ▼ 

Germany 

542 ▼ 

533 ▼ 

546 

530 ▼ 

Slovenia 

541 ▼ 

544 

547 

539 ▼ 

Slovak Republic 

539 ▼ 

531 ▼ 

538 

531 ▼ 

Macao-CHN 

536 ▼ 

556 O 

549 

543 ▼ 

Israel 2 

532 ▼ 

529 ▼ 

530 ▼ 

530 ▼ 

Spain 

530 ▼ 

527 ▼ 

526 ▼ 

529 ▼ 

Portugal 2 

528 ▼ 

528 ▼ 

528 ▼ 

526 ▼ 

Kazakhstan 

527 ▼ 

544 

529 ▼ 

542 ▼ 

New Zealand 

525 ▼ 

520 ▼ 

521 ▼ 

525 ▼ 

Belgium (Flemish)-BEL 

524 ▼ 

526 ▼ 

526 ▼ 

524 ▼ 

France 

513 ▼ 

510 ▼ 

521 ▼ 

501 ▼ 

Belgium (French)-BEL 2 

504 ▼ 

490 ▼ 

501 ▼ 

494 ▼ 

Chile 

500 ▼ 

485 ▼ 

496 ▼ 

491 ▼ 

Georgia 2 

490 ▼ 

486 ▼ 

486 ▼ 

490 ▼ 

Trinidad and Tobago 

478 ▼ 

480 ▼ 

483 ▼ 

472 ▼ 

Azerbaijan 

466 ▼ 

477 ▼ 

477 ▼ 

465 ▼ 


See notes at the end of the table. 
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PIRLS Tables and Figures 


Table 2. PIRLS purposes for reading and processes of comprehension average subscale scores of fourth-grade students, by 
education system: 2016—Continued 


Purposes for reading_ _ Processes of comprehension 


Education system 

Literary experience 

Acquire and use 
information 

Retrieving and 
straightforward 
inferencing 

Interpreting, 
integrating, and 
evaluating 

PIRLS scale centerpoint 1 

500 

500 

500 

500 

Malta 2 

452 ▼ 

451 ▼ 

452 ▼ 

451 ▼ 

United Arab Emirates 

440 ▼ 

460 ▼ 

448 ▼ 

453 ▼ 

Bahrain 

437 ▼ 

453 ▼ 

444 ▼ 

446 ▼ 

Qatar 

434 ▼ 

450 ▼ 

442 ▼ 

441 ▼ 

Iran, Islamic Republic of 

430 ▼ 

425 ▼ 

429 ▼ 

425 ▼ 

Saudi Arabia 

430 ▼ 

429 ▼ 

425 ▼ 

439 ▼ 

Oman 

411 ▼ 

425 ▼ 

419 ▼ 

415 ▼ 

Kuwait 

388 ▼ 

398 ▼ 

394 ▼ 

388 ▼ 

Morocco 

353 ▼ 

359 ▼ 

364 ▼ 

336 ▼ 

Egypt 

328 ▼ 

332 ▼ 

329 ▼ 

340 ▼ 

South Africa 

323 ▼ 

314 ▼ 

321 ▼ 

308 ▼ 

Benchmarking education systems 

Moscow City-RUS 

613 O 

613 O 

611 O 

614 O 

Madrid-ESP 2 

550 

548 

547 

550 

Quebec-CAN 2 

550 

547 

551 

545 ▼ 

Ontario-CAN 

549 

539 

539 

548 

Andalusia-ESP 

526 ▼ 

524 ▼ 

522 ▼ 

527 ▼ 

Dubai-UAE 

508 ▼ 

523 ▼ 

512 ▼ 

519 ▼ 

Buenos Aires-ARG 

484 ▼ 

475 ▼ 

483 ▼ 

473 ▼ 

Abu Dhabi-UAE 

406 ▼ 

422 ▼ 

413 ▼ 

417 ▼ 


© Score is higher than U.S. average score. 

▼ Score is lower than U.S. average score. 

1 The PIRLS scale centerpoint is set at 500 points and represents the mean of the overall achievement distribution in 2001. The PIRLS scale is the 
same in each administration; thus a value of 500 in 2016 equals 500 in 2001. 

2 See appendix table A-l for details on sampling issues in these education systems. 

NOTE: Education systems are ordered by literary experience average subscale score. Italics indicate participants identified and counted in this report 
as a nonnational entity that represents a portion of a country. All average scores reported as higher or lower than the U.S. average score are differ¬ 
ent at the .05 level of statistical significance. The tests for significance take into account the standard error for the reported difference. Thus, a small 
difference between the United States and one education system may be significant, while a large apparent difference between the United States and 
another education system may not be significant. The standard errors of the estimates are shown in table 2 available at https://nces.ed.gov/pirls/ 
pirls2016/pirls2016 table02.asp . Education systems that did not administer PIRLS at the target grade are not shown; see the international report for 
their results. Five education systems participated in PIRLS Literacy (Egypt, Iran, Kuwait, Morocco, and South Africa); two of these education systems 
completed both PIRLS and PIRLS Literacy (Iran and Morocco). More detail on PIRLS Literacy is available in the technical notes at https://nces.ed.gov/ 
pirls/pirlsl6technotes.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 
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PIRLS Tables and Figures 


Figure 2. Change in PIRLS overall reading average scale scores of fourth-grade students, by education system: 2011 to 2016 
and 2001 to 2016 


Average Score 


2001 

2011 

2016 

Education system 

528 ▼ 

568 © 

581 O 

Russian Federation 1 

528 ▼ 

567 O 

576 O 

Singapore 12 

528 ▼ 

571 © 

569 © 

Hong Kong-CHN 12 

553 O 

552 

559 O 

England-GBR 1 

561 O 

542 ▼ 

555 

Sweden 

543 

539 ▼ 

554 

Hungary 

550 

532 ▼ 

552 

Bulgaria 

543 

528 ▼ 

550 

Lithuania 13 

542 

556 

549 

United States 12 

541 

541 ▼ 

548 

Italy 

554 O 

546 ▼ 

545 

Netherlands 12 

537 

545 ▼ 

543 

Czech Republic 

502 ▼ 

530 ▼ 

542 

Slovenia 

539 

541 ▼ 

537 ▼ 

Germany 

518 ▼ 

535 ▼ 

535 ▼ 

Slovak Republic 

529 ▼ 

531 ▼ 

523 ▼ 

New Zealand 

499 ▼ 

507 ▼ 

517 ▼ 

Norway (4) 14 

525 ▼ 

520 ▼ 

511 ▼ 

France 

414 ▼ 

457 ▼ 

428 ▼ 

Iran, Islamic Republic of 



See notes at the end of the figure. 
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PIRLS Tables and Figures 


Figure 2. Change in PIRLS overall reading average scale scores of fourth-grade students, by education system: 2011 to 2016 
and 2001 to 2016—Continued 


2001 

Average Score 

2011 

2016 

Education system 

Change in average score 




Benchmarking education systems 






^■ 10 ! 

537 

538 ▼ 

547 

Quebec-CAN 12 

10 






548 

552 

544 

Ontario-CAN 1 





l 

-40 

1 1 1 1 1 

-20 0 20 40 60 





Score difference 


■ Change from 2011 to 2016 ■ Change from 2001 to 2016 

© Score is higher than U.S. average score in the same year. 

▼ Score is lower than U.S. average score in the same year. 

# Rounds to zero. 

*p < .05. Change in average scores is statistically significant. 

1 See appendix table A-2 for details on sampling issues in these education systems in 2001 and 2011. 

2 See appendix table A-l for details on sampling issues in these education systems in 2016. 

3 Trend results for Lithuania do not include students taught in Polish or Russian. 

4 The number in parentheses indicates the grade level. For PIRLS 2016, Norway revised its assessed population to students in the fifth grade to obtain 
better comparisons with Sweden and Finland. However, in previous PIRLS cycles Norway assessed students in the fourth grade, which is similar to 
third grade in many other education systems because grade 1 in Norway is considered the equivalent of a year of kindergarten. To maintain trend with 
previous PIRLS cycles, in 2016 Norway also collected data from fourth-grade students, which is used in this trend table. 

NOTE: Education systems are ordered by 2016 overall average scale scores. Italics indicate participants identified and counted in this report as a 
nonnational entity that represents a portion of a country. Data are not shown for some education systems because comparable data from previous 
cycles are not available. All average scores reported as higher or lower than the U.S. average score are different at the .05 level of statistical 
significance. The tests for significance take into account the standard error for the reported difference. Thus, a small difference between the United 
States and one education system may be significant, while a large difference between the United States and another education system may not 
be significant. Detail may not sum to totals because of rounding. The standard errors of the estimates are shown in figure 2 available at https:// 
nces.ed.gov/pirls/pirls2016/pirls2016 figure02.asp . Education systems that did not administer PIRLS at the target grade are not shown; see the 
international report for their results. In 2016, Iran participated in both PIRLS and PIRLS Literacy. More detail on PIRLS Literacy is available in the 
technical notes at https://nces.ed.gov/pirls/pirlsl6technotes.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2001, 
2011, and 2016. 
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Figure 3. Trends in U.S. fourth-grade students’ PIRLS overall reading average scale scores and cut scores at the 10th, 25th, 
75th, and 90th percentiles: 2001,2006,2011, and 2016 


1,000 


650 - 


600 - 


550 - 


500 - 


450 


0 





*p < .05. Score is significantly different from 2016 score. 

NOTE: See appendix tables A-l and A-2 for details on coverage and sampling issues in the United States for 2016 and earlier years, respectively. The 
standard errors of the estimates are shown in table 8 available at https://nces.ed.gov/pirls/pirls2016/pirls2016 table08.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2001, 
2006, 2011, and 2016. 
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PIRLS Tables and Figures 


Figure 4. PIRLS overall reading average scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and percentage of 
public school students eligible for free or reduced-price lunch: 2016 


U.S. average score 


549 
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0 
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o 

0 
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White 
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Hispanic 
Asian 
All other races 


(571* 

518* 

1525* 

^^^■591 
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"D 

0 
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0 <- 
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0 
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Less than 10 percent 
10 to 24.9 percent 
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50 to 74.9 percent 
75 percent or more 


0 


587* 

592* 

566* 


546 


516* 
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T 


// 


100 


200 


300 400 

Scale score 


500 


600 


1,000 


*p < .05. Significantly different from the U.S. average score. 

NOTE: The U.S. met guidelines for sample participation rates only after replacement schools were included. Black includes African American, and 
Hispanic includes Latino. All other races includes American Indian or Alaskan Native; Native Hawaiian or other Pacific Islander; or Two or more races. 
Racial categories exclude Hispanic origin. Students who identified themselves as being of Hispanic origin were classified as Hispanic, regardless of 
their race. Analyses for free or reduced-price lunch are limited to public schools only, based on school reports of the percentage of students in public 
school eligible for the federal free or reduced-price lunch program. The standard errors of the estimates are shown in table 10 available at https:// 
nces.ed.gov/pirls/pirls2016/pirls2016 tablelO.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 
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ePIRLS Tables and Figures 


Table 3. ePIRLS online informational reading average scale scores of fourth-grade students, by education system: 2016 


Education system 

Online average 
scale score 

Education system 

Online average 
scale score 

ePIRLS scale centerpoint 1 

500 

ePIRLS scale centerpoint 1 

500 

Singapore 2 

588 O 

Italy 

532 ▼ 

Norway 

568 O 

Slovenia 

525 T 

Ireland 

567 O 

Portugal 2 

522 T 

Sweden 

559 

Georgia 2 

477 T 

Denmark 2 

558 

United Arab Emirates 

468 T 

United States 2 

557 



Chinese Taipei-CHN 

546 v 

Benchmarking education systems 


Canada 2 

543 ▼ 

Dubai-UAE 

528 T 

Israel 2 

536 v 

Abu Dhabi-UAE 

431 v 


© Score is higher than U.S. average score. 

▼ Score is lower than U.S. average score. 

1 The ePIRLS scale centerpoint is set at 500 points and represents the mean of the overall PIRLS achievement distribution in 2001. ePIRLS results 
are reported on the PIRLS reading achievement scale. 

2 See appendix table A-l for details on sampling issues in these education systems. 

NOTE: Education systems are ordered by online average scale score. Italics indicate participants identified and counted in this report as a nonna¬ 
tional entity that represents a portion of a country. All average scores reported as higher or lower than the U.S. average score are different at the .05 
level of statistical significance. The tests for significance take into account the standard error for the reported difference. Thus, a small difference 
between the United States and one education system may be significant while a large apparent difference between the United States and another 
education system may not be significant. The standard errors of the estimates are shown in table 11 available at https://nces.ed.gov/pirls/pirls2016/ 
pirls2016 tablell.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 

2016. 
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Figure 5. Percentage of fourth-grade students reaching the ePIRLS international benchmarks in online informational reading, 
by education system: 2016 


Education system 




Percent of students reaching each international benchmark 

Advanced 

(625) 

High 

(550) 

Intermediate 

(475) 

Low 

(400) 






72 O 

92 O 
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96 ▼ 






51 T 

86 

97 






41 T 

82 T 

98 






39 T 

78 ▼ 

95 ▼ 
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85 T 






50 T 

84 

97 

Benchmarking education 








systems 













44 T 
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91 ▼ 
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36 T 

61 ▼ 
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0 

20 

40 

60 

80 100 






Percent 






| Advanced | High ■ Intermediate | Low ■ Below Low 
O Percentage is higher than the U.S. percentage at the same benchmark. 

▼ Percentage is lower than the U.S. percentage at the same benchmark. 

1 See appendix table A-l for details on sampling issues in these education systems. 

NOTE: Education systems are ordered by the percentage of students reaching the Advanced international benchmark. Italics indicate participants 
identified and counted in this report as a nonnational entity that represents a portion of a country. All percentages reported as higher or lower 
than the U.S. percentage are different at the .05 level of statistical significance. The tests for significance take into account the standard error for 
the reported difference. Thus, a small difference between the United States and one education system may be significant while a large difference 
between the United States and another education system may not be significant. The international median represents all participating ePIRLS 
education systems, including the United States. The international median represents the percentage at which half of the education systems have 
that percentage of students at or above the median and half have that percentage of students below the median. Benchmarking participants are not 
included in the international median. The standard errors of the estimates are shown in figure 3 available at https://nces.ed.gov/pirls/pirls2016/ 
pirls2016 figure03.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 
2016. 
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Figure 6. ePIRLS online informational reading average scale scores of U.S. fourth-grade students, by sex, race/ethnicity, and 
percentage of public school students eligible for free or reduced-price lunch: 2016 
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*p < .05. Significantly different from the U.S. average score. 

NOTE: The U.S. met guidelines for sample participation rates only after replacement schools were included. Black includes African American, and 
Hispanic includes Latino. All other races includes American Indian or Alaskan Native; Native Hawaiian or other Pacific Islander; or Two or more races. 
Racial categories exclude Hispanic origin. Students who identified themselves as being of Hispanic origin were classified as Hispanic, regardless of 
their race. Analyses for free or reduced-price lunch are limited to public schools only, based on school reports of the percentage of students in public 
school eligible for the federal free or reduced-price lunch program. The standard errors of the estimates are shown in table 15 available at https:// 
nces.ed.gov/pirls/pirls2016/pirls2016 table!5.asp . 

SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 
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Appendix A. Technical Notes 


This appendix briefly describes features of the PIRLS 2016 assessment, with a 
particular focus on its implementation in the United States. For further details about 
the assessment and any of the topics discussed here, see the fuller description of 
technical notes on the NCES PIRLS website at https://nces.ed.gov/surveys/pirls/ 
pirls2016results.asp as well as the lEA's PIRLS 2016 Technical Report (Martin, Mullis, 
and Hooper 2017). 

Sampling and Response Rates 

The IEA required all participating education systems to adhere to the PIRLS 2016 
technical standards (Martin, Mullis, and Hooper 2017), which provided detailed 
information about the target population, sampling, response rates, translation, 
assessment administration, and data submission. All students enrolled in the grade 
that represents the fourth year of formal schooling constitute the desired international 
target population, provided that their mean age at the time of testing is at least 9.5 
years and that the grade level is calculated from the first year of the International 
Standard Classification of Education (ISCED) Level 1. (ISCED Level 1 is the equivalent 
of grade 1 in the United States.) For most education systems, the target grade is grade 
4 or its national equivalent. All students enrolled in the target grade, regardless of their 
age, belong to the desired international target population. Additional detail can be 
found in the detailed technical notes, available online at https://timssandpirls.bc.edu/ 
publications/pirls/2016-methods.html . 

A representative sample of fourth-grade students was selected using a stratified 
two-stage systematic sample, with sampling probabilities proportional to the 
estimated number of fourth-grade students in the school based on grade enrollments. 
Classrooms within sampled schools were selected using sampling software provided by 
the TIMSS & PIRLS International Study Center. 

PIRLS guidelines call for a minimum of 150 schools to be sampled, with a minimum 
of 4,000 students assessed. The U.S. PIRLS 2016 national school sample consisted 
of 176 schools, which was higher than the international sampling minimum of 150 
to offset anticipated school nonresponse and ineligibility. A total of 158 U.S. schools 
agreed to participate in PIRLS 2016, including 131 from the original sample and 27 
sampled as replacements for nonparticipating schools from the original sample. Of 
the 158 U.S. schools that participated in PIRLS, 153 also participated in ePIRLS. In 
total, 4,425 U.S. students participated in PIRLS and 4,090 of these students also 
participated in ePIRLS. 
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The overall weighted school response rate was 92 percent for PIRLS and 89 percent for ePIRLS. The 
overall weighted student response rate was 94 percent for PIRLS and 90 percent for ePIRLS, and the 
overall U.S. student exclusion rate was 4.8 percent for PIRLS and 4.9 percent for ePIRLS. Additionally, 
as required by NCES standards, a nonresponse bias analysis was conducted because the U.S. school- 
level response rate (before substitution) for PIRLS fell below 85 percent. The purpose of this analysis 
was to examine whether the participation status of schools was related to various characteristics 
and thus introduced the potential for bias in the results. The results suggested some potential for 
nonresponse bias in the U.S. sample (prior to substitution) based on the characteristics studied. 

It also suggested that the use of substitute schools reduced the potential for bias in all variables 
except Census region, such that schools in the Central Region are represented at higher rates among 
participating schools relative to all eligible schools. Nonresponse bias results for the final sample 
(after substitution) indicated that schools with larger fourth-grade enrollment were more likely to 
respond than those with smaller fourth-grade enrollment. Applying nonresponse adjustments lowered 
estimates of bias, relative bias, and effect size associated with bias. After the application of school 
nonresponse adjustments, bias estimates remained on a few variables; however, the magnitudes 
of these bias estimates were quite small. Variables that indicated bias include Census region and 
percentage school enrollment of American Indian or Alaska Native students. 

As indicated by the footnotes in the cross-education system tables and figures in this report, there 
were sampling or other issues in the United States and 19 education systems. For the current 
administration, these specific issues are detailed in table A-l. For the 2001, 2006, and 2011 
administrations, these issues are addressed in table A-2. 

See https://timssandpirls.bc.edu/publications/pirls/2016-methods.html for the PIRLS international 
sampling guidelines and requirements regarding accommodations, exclusions, and response rate 
requirements, as well as the response rates of all participating education systems. 
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Table A-1. Sampling or other issues for PIRLS and ePIRLS, by education system: 2016 
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SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 
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Table A-2. Sampling or other issues for PIRLS, by education system: 2001,2006, and 2011 
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SOURCE: International Association for the Evaluation of Educational Achievement (IEA), Progress in International Reading Literacy Study (PIRLS), 2016. 
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Assessment and Questionnaires 

The 2016 assessment instruments were developed by international experts and PIRLS international 
test developers; the instruments included items submitted by participating education systems. Each 
reading passage and its accompanying items were assigned to a block and then systematically 
distributed among individual student assessment booklets. The 2016 assessment consisted of 
12 passages or blocks, distributed across 15 booklets (with questions printed after each passage) 
and one reader (presented in a magazine-type format with the questions in a separate booklet). 

Six blocks were included from previous PIRLS assessments to provide a link for measuring trends 
in reading achievement. Approximately 39 percent of assessment score points were based on 
multiple choice questions and 61 percent of the assessment score points were based on open 
ended questions. Open-ended items were coded by trained scorers following international coding 
guidelines. 

All participating education systems field-tested the assessment instruments in spring 2015. After 
the field test, passages and items were dropped if they did not meet the established measurement 
criteria or were otherwise found to include intrinsic biases. Dropped passages and items were not 
included in the main assessment. 

For the main assessment, each student received a booklet containing two reading passages: one 
literary experience passage and one informational passage. Students had 40 minutes to complete 
each passage and all questions about that passage, with a 5- to 10-minute break between each 
passage. Then they received a questionnaire asking about their backgrounds, attitudes, and 
experiences in school. Students had 30 minutes to complete the questionnaire. 

The ePIRLS assessment presented students with a simulated internet environment and five possible 
tasks. Typically, students who participated in the ePIRLS assessment took the assessment on the 
day following PIRLS. Like PIRLS, ePIRLS was delivered in two 40-minute segments with a 5-minute 
break between them, followed by a brief questionnaire about computer use. 

As part of the PIRLS study, principals were asked to complete a questionnaire about their school's 
structure, resources, instruction, climate, and policies. Language Arts teachers of assessed 
classrooms were also asked to complete a questionnaire about their experience, available resources, 
and instructional practices at the school. 

See https://nces.ed.gov/surveys/pirls/pirls2016results.asp for more information about the PIRLS 
2016 assessment design and questionnaires. 
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Reporting Results 

In PIRLS 2016, results are generally reported in two ways: scale scores and international 
benchmarks of achievement. Scores on PIRLS for each administration since 2001 have been scaled 
to range from 0 to 1,000, with an international centerpoint of 500 and a standard deviation of 100. 
By centering results in this manner, comparisons can be made from 2001 to 2006, to 2011, and 
now to 2016. The ePIRLS scale also ranges from 0 to 1,000, with an international centerpoint of 
500 and a standard deviation of 100. This scaling facilitates comparisons across education systems 
of overall printed reading achievement and overall online reading achievement. 

Along with scale scores, PIRLS has international benchmarks that group achievement into four 
levels: Advanced, High, Intermediate, and Low. The distribution of student scores—and the kinds 
of skills and knowledge that students demonstrate—determines the score cut-points for these 
benchmarks. The benchmarks offer an interpretation of what the scale scores mean using scale 
anchoring. Scale anchoring involves selecting benchmarks on the PIRLS achievement scales to be 
described in relation to student performance. Items within each of these benchmarks identify what 
students are likely to answer correctly; experts then examine the data to provide a sense of what 
students know and can do. Examples of items at each benchmark level are provided at https:// 
timssandpirls.bc.edu/pirls2016/frameworks.html . 

Sampling Weights and Standard Errors 

The use of sampling weights is necessary to compute statistically sound estimates. Survey weights 
adjust for the probability of selection of individual schools and students, for school or student 
nonresponse, and for errors in estimating the size of a school or the number of fourth-grade 
students in a school at the time of sampling. As with any study, estimates produced using data from 
PIRLS 2016 are subject to two types of error: nonsampling errors and sampling errors. The sources 
of nonsampling errors are typically problems such as unit and item nonresponse, differences in 
respondents’ interpretations of the meaning of survey questions, and mistakes in data preparation. 
Sampling errors arise when a sample of the population, rather than the whole population, is used 
to estimate some statistic. Different samples from the same population would likely produce 
somewhat different estimates of the statistic in question. This uncertainty is referred to as sampling 
variance and is usually expressed as the standard error of a statistic estimated from sample data. 
Standard errors for all statistics reported in this report are available online at https://nces.ed.gov/ 
surveys/pirls/pirls2016results.asp . 
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Statistical Comparisons 

The comparisons made in this report have been tested for statistical significance. For example, in the 
commonly made comparison of international averages to U.S. averages, tests of statistical significance 
were used to establish whether or not the observed differences from the U.S. average were statistically 
significant. In all instances, the tests for significance used were standard t tests. These fell into two 
categories according to the nature of the comparison being made: comparisons of independent 
samples and comparisons of nonindependent samples. A difference is “significant” if the probability 
associated with the t test is less than .05. If a test is significant, it implies that the difference in the 
observed value in the sample represents a real difference in the population. No adjustments were 
made for multiple comparisons. 

More information on statistical tests and more detailed information on the implementation of 
PIRLS in the United States are available on the NCES website at https://nces.ed.gov/surveys/pirls/ 
pirls2016results.asp . 
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